-- Retinal vein cannulation (RVC) is a minimally invasive microsurgical procedure for treating retinal vein occlusion (RVO), a leading cause of vision impairment. However, the small size and fragility of retinal veins, coupled with the need for high-precision, tremor-free needle manipulation, create significant technical challenges. These limitations highlight the need for robotic assistance to improve accuracy and stability. This study presents an automated robotic system with a top-down microscope and B-scan optical coherence tomography (OCT) imaging for precise depth sensing. Deep learning-based models enable real-time needle navigation, contact detection, and vein puncture recognition, using a chicken embryo model as a surrogate for human retinal veins. The experiments demonstrate notable reductions in navigation and puncture times compared to manual methods. Our results demonstrate the potential of integrating advanced imaging and deep learning to automate microsurgical tasks, providing a pathway for safer and more reliable RVC procedures with enhanced precision and reproducibility. I. INTRODUCTION Retinal vein occlusion (RVO) occurs due to the blockage of a retinal vein by a thrombus, leading to transient or permanent vision loss [1]. Current treatments focus on managing complications, but no standardized surgical approach exists for thrombus removal. A 2015 meta-analysis identified RVO as the second most prevalent retinal vascular disease globally, affecting 28.06 million people aged 30-89, including 23.38 million branch RVO (BRVO) and 4.67 million central RVO (CRVO) [2]. Retinal vein cannulation (RVC) involves inserting a micro-needle into the occluded retinal vein, followed by injecting a thrombolytic agent to dissolve the clot [3].

Recent advancements in retinal surgery have paved the way for a modern operating room equipped with a surgical robot, a microscope, and intraoperative optical coherence tomography (iOCT)- a depth sensor widely used in retinal surgery. Integrating these tools raises the fundamental question of how to effectively combine them to enable surgical autonomy. In this work, we tackle this question by developing a unified framework that facilitates real-time autonomous surgical workflows leveraging these devices. The system features: (1) a novel imaging system that integrates the microscope and iOCT in real-time by dynamically tracking the surgical instrument via a small iOCT scanning region, providing real-time depth feedback; (2) implementation of convolutional neural networks (CNN) that automatically detect and segment task-relevant information for surgical autonomy; (3) intuitive selection of goal waypoints within both the microscope and iOCT views through simple mouse-click interactions; and (4) integration of model predictive control (MPC) for trajectory generation, ensuring patient safety by implementing safety-related kinematic constraints. The system's utility is demonstrated by automating subretinal injection (SI), a challenging procedure with high accuracy and depth perception requirements. We validate our system by conducting 30 successful SI trials on pig eyes, achieving mean needle insertion accuracy of 26 micrometers to various subretinal goals and mean duration of 55 seconds. Preliminary comparisons to a human operator performing SI in robot-assisted mode highlight the enhanced safety of our system. Project website is here: https://sites.google.com/view/eyesurgerymicroscopeoct/home

The consumption of microbial-contaminated food and water is responsible for the deaths of millions of people annually. Smartphone-based microscopy systems are portable, low-cost, and more accessible alternatives for the detection of Giardia and Cryptosporidium than traditional brightfield microscopes. However, the images from smartphone microscopes are noisier and require manual cyst identification by trained technicians, usually unavailable in resource-limited settings. Automatic detection of (oo)cysts using deep-learning-based object detection could offer a solution for this limitation. We evaluate the performance of three state-of-the-art object detectors to detect (oo)cysts of Giardia and Cryptosporidium on a custom dataset that includes both smartphone and brightfield microscopic images from vegetable samples. Faster RCNN, RetinaNet, and you only look once (YOLOv8s) deep-learning models were employed to explore their efficacy and limitations. Our results show that while the deep-learning models perform better with the brightfield microscopy image dataset than the smartphone microscopy image dataset, the smartphone microscopy predictions are still comparable to the prediction performance of non-experts.

Summary: A novel AI system recognizes a neuron in microscope images of the brain more efficiently than any previous approach. The system improves current methods of tracing neurons and their connections, which can help improve brain mapping. Cold Spring Harbor Laboratory (CSHL) scientists have taught computers to recognize a neuron in microscope images of the brain more efficiently than any previous approach. The researchers improved the efficiency of automated methods for tracing neurons and their connections, a task that is increasingly in demand as researchers work to map the brain's densely interconnected circuits. They did it by teaching the computer to recognize different parts of neurons, each of which have different characteristics. Such connection maps are critical for learning how the brain processes information to generate thoughts and behavior.

We extend the blindspot model for self-supervised denoising to handle Poisson-Gaussian noise and introduce an improved training scheme that avoids hyperparameters and adapts the denoiser to the test data. Self-supervised models for denoising learn to denoise from only noisy data and do not require corresponding clean images, which are difficult or impossible to acquire in some application areas of interest such as low-light microscopy. We introduce a new training strategy to handle Poisson-Gaussian noise which is the standard noise model for microscope images. Our new strategy eliminates hyperparameters from the loss function, which is important in a self-supervised regime where no ground truth data is available to guide hyperparameter tuning. We show how our denoiser can be adapted to the test data to improve performance. Our evaluation on a microscope image denoising benchmark validates our approach.

An illustration representing Deep-Z, an artificial intelligence-based framework that can digitally refocus a 2D fluorescence microscope image (at bottom) to produce 3D slices (at left). A University of California, Los Angeles research team has devised a technique that extends the capabilities of fluorescence microscopy, which allows scientists to precisely label parts of living cells and tissue with dyes that glow under special lighting. The researchers use artificial intelligence to turn two-dimensional images into stacks of virtual three-dimensional slices showing activity inside organisms. In a study published in Nature Methods on November 4, 2019, the scientists also reported that their framework, called "Deep-Z," was able to fix errors or aberrations in images, such as when a sample is tilted or curved. Further, they demonstrated that the system could take 2D images from one type of microscope and virtually create 3D images of the sample as if they were obtained by another, more advanced microscope.

A UCLA research team has devised a technique that extends the capabilities of fluorescence microscopy, which allows scientists to precisely label parts of living cells and tissue with dyes that glow under special lighting. The researchers use artificial intelligence to turn two-dimensional images into stacks of virtual three-dimensional slices showing activity inside organisms. In a study published in Nature Methods, the scientists also reported that their framework, called "Deep-Z," was able to fix errors or aberrations in images, such as when a sample is tilted or curved. Further, they demonstrated that the system could take 2-D images from one type of microscope and virtually create 3-D images of the sample as if they were obtained by another, more advanced microscope. "This is a very powerful new method that is enabled by deep learning to perform 3-D imaging of live specimens, with the least exposure to light, which can be toxic to samples," said senior author Aydogan Ozcan, UCLA chancellor's professor of electrical and computer engineering and associate director of the California NanoSystems Institute at UCLA.

A version of this story appeared in STAT Health Tech, our weekly newsletter about how tech is transforming health care and the life sciences. Sign up here to receive it in your inbox. You're looking at two versions of the same video of a moment in a single cell, captured under a powerful microscope. The red and yellow structures are mitochondria, and the inset magnified in the bottom left hand corner in each view captures a mitochondrion dividing. The view on the left shows the raw data as it came off the microscope; you might think of it like a social media influencer's first take, before any filters have been applied to get that Instagram-ready look.

How do you study a blood cell if you can't see it? You feel it, using a device that translates scientific data into tactile information. "We want to help people that are blind or visually impaired and studying science," says Ting Zhang at Purdue University in Indiana. Her team has developed a system that uses a haptic device – an interface that gives you feedback you can touch or physically feel – to let people interpret visual information using their hands. This sophisticated joystick is hooked up to a computer connected to a microscope.